1. Field of the Invention
This invention pertains to a moving body recognition system for enabling a computer to analyze the movement of an object based on an image e.g. captured by a television camera, and more particularly to a moving body recognition apparatus for recognizing the shape (the coordinate of a feature point) and movement (rotation) of an object moving on a plane coupled with a rotation.
2Description of the Related Arts
A moving body recognition apparatus is for recognizing an object by digitally processing image signals of an externally moving object captured by a television camera. Moving body recognition apparatuses are used widely, e.g. for an FA (factory automation) inspection, an automatic monitoring device, and a visual sensor for an automatically operated vehicle, generally as devices capable of processing visual information as with a human being.
A moving body recognition apparatus generally recognizes the shape of a moving body and its relative movement, when an image input unit of a moving body recognition apparatus is moving in relation to the object. That is, even if an object is not actually moving, when an image input unit of a moving body recognition apparatus moves, the moving body recognition apparatus recognizes the shape of the object standing still and the movement of the image input unit. For example, in a case of a visual sensor for an automatically operated vehicle, its image input unit is loaded on top of the vehicle, and the moving body recognition apparatus recognizes the environment in which the vehicle is running.
A moving body recognition apparatus must be compact and responsive, since their uses are wide spread. Compactness is critical, especially when a moving body recognition apparatus is loaded on a vehicle in its application to a visual sensor of an automatically operated vehicle. Responsiveness is crucial, because a realtime processing similar to a human vision is required.
A conventional moving body recognition device captures an object by two [2] image input units. By establishing the correspondences between feature points of an object in the two [2] images captured by the two [2] image input units, the shape of the object is captured at every certain instant in time for observation by applying a principle of a triangulation, and then the movement of the object is calculated.
FIG. 1 shows a concept of a conventional moving body recognition apparatus.
In FIG. 1, 1 is an object and a black blot represents a feature point. Feature points represent particular positions of an object, such as peaks, peripheral points and pattern points in case of the presence of a pattern and color boundaries.
A first image input unit 2 and a second image input unit 3 each comprise a television camera. A moving body recognition unit 4 receives two [2] images from the first image input unit 2 and the second image input unit 3. The moving body recognition unit 4 detects feature points of an object 1 from the two [2] images. By matching the same feature point between the two [2] images, the position of a feature point is calculated by applying a principle of a triangulation.
The moving body recognition unit 4 calculates the movement of a feature point and the object 1 from a shift of the feature point in a time series. The moving body recognition apparatus has a recognition result output unit 5 output the position and movement of a feature point and the movement of an object.
FIG. 2 shows a configuration of a conventional moving body recognition apparatus.
Parts shown in FIG. 2 which are the same as those shown in FIG. 1 have the same numbers.
Feature point 1 extraction unit 10 extracts a feature point from an image inputted from the first image input unit 2 and supplies it to a feature point correspondence unit 12. Likewise, a second feature point extraction unit 11 extracts a feature point from an image inputted from the second image input unit 3 and supplies it to the feature point correspondence unit 12. The feature point correspondence unit 12 matches the same feature points from among the feature points extracted from feature point 1 extraction unit and the second feature point extraction unit 11.
A feature point position calculation unit 13 obtains the positions of feature points by relating the positions of the matched feature points with the positions of the first image input unit 2 and the second image input unit 3, and stores the result in a feature point position storage unit 14. The positions of feature points at plural instants in time for observation stored in the feature point position storage unit 14 are sent to an object movement calculation unit 15, which calculates the movement of an object and stores the result in an object movement storage unit 16.
However, a conventional moving body recognition apparatus as explained in the description of FIGS. 1 and 2 have the following problems.
(a) Because two [2] feature point extraction units need to individually extract feature points from two
[2] images captured respectively by two [2] image input units, the process load for extracting a feature point is twice as much as that by using a single television camera.
(b) An additional process of matching feature points from two [2] images captured differently is required. The feature point correspondence unit 12 is required as shown in FIG. 2. Because the positions of two [2] image input units are different, they capture the object 1 differently. This makes it difficult to match feature points of the object 1. The closer the positions of the two [2] image input units 2 and 3, the easier it is to establish correspondences between feature points of the object 1, but the less accurate a recognition of the shape of the object 1 becomes. If, on the other hand, the two [2] image input units 2 and 3 are spaced apart for improving the precision in recognizing the object 1, the feature point correspondence unit 12 would require a large processing time for searching for corresponding feature points.
Typically, processing in (b) is impossible for a conventional moving body recognition apparatus, when a feature point of the object 1 captured by one [1] of the two [2] image input units 2 and 3 cannot be captured by the other, in which case no correspondence between those feature points can be established.
FIG. 3 shows an example in which a conventional moving body recognition apparatus fails to establish correspondences between feature points captured by different image input units.
The object 1 has two [2] feature points, for instance. However, there is a case in which both the first image input unit 2 and the second image input unit 3 can capture only one [1] of the two [2] feature points.